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ABSTRACT 

A human-in-the-loop simulation investigated the robustness of a schedule-based 
terminal-area air traffic management concept, and its supporting controller tools, to 
off-nominal events - events that led to situations in which runway arrival schedules 
required adjustments and controllers could no longer use speed control alone to 
impose the necessary delays. The main research question was exploratory: to assess 
whether controllers could safely resolve and control the traffic during off-nominal 
events. A focus was the role of the supervisor - how he managed the schedules, 
how he assisted the controllers, what strategies he used, and which combinations of 
tools he used. Observations and questionnaire responses revealed supervisor 
strategies for resolving events followed a similar pattern: a standard approach 
specific to each type of event often resolved to a smooth conclusion. However, due 
to the range of factors influencing the event (e.g., environmental conditions, aircraft 
density on the schedule, etc.), sometimes the plan required revision and actions had 
a wide-ranging effect. 
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INTRODUCTION 


In today’s air traffic system, control in a busy terminal area can include multiple 
speed changes, altitude level-offs, and/or heading vectors. In periods of heavy 
demand, this very safe system works hard to maintain high throughput, but does so 
at the expense of flight efficiency. That efficiency is commonly traded for positive 
control, so that controllers can quickly deliver aircraft from one sector to the next 
with proper spacing that allows aircraft to safely descend for landing. When demand 
is low, there is more space between aircraft and therefore less reason to be 
concerned with compression; less controller intervention is needed and aircraft can 
more easily descend along an efficient descent profile. 

The Super-Density Operations research under NASA’s Airspace Systems 
Program aims to safely sustain high runway throughput while still accommodating 
fuel-efficient operations. Advanced scheduling capabilities create schedules at the 
runway to enable Optimized Profile Descents (OPDs) along Area Navigation 
(RNAV) routes. Assuming en route controllers feed the Terminal Radar Approach 
Control (TRACON) with reasonable schedule errors, TRACON controllers can 
avoid costly altitude and heading maneuvers and instead rely primarily on speed 
adjustments to minimize runway schedule conformance errors (Isaacson, Robinson, 
Swenson, & Denery, 2010). 

Controller-Managed Spacing (CMS) research in the Airspace Operations 
Laboratory (AOL; Prevot, et al., 2010) has used a series of real-time human-in-the- 
loop simulations to investigate specific controller decision support tools (DSTs) for 
such operations. With relatively straight-forward display enhancements, TRACON 
controllers were able to manage dense arrival flows that followed OPDs along 
RNAV routes and met runway schedule times (see Kupfer, Callantine, Martin, 
Mercer & Palmer, 2011; Callantine, Palmer & Kupfer, 2010). 

Having tested the CMS concept and tools under conditions with only speed 
variances, in 2011 research examined how robust the concept was to significant 
disturbances and off-nominal conditions. Accepting that large disturbances to 
operations in the TRACON are an eventuality, the response and recovery to an off- 
nominal event also requires investigation. In particular, the research sought to 
investigate whether the CMS concept and tools can support the response to and 
recovery from a disturbance (see Callantine, Cabrall, Kupfer, Martin, Mercer & 
Palmer, 2011). 

This paper will focus on the role of the supervisor, who managed the runway 
schedule as part of the controller team’s response to scripted off-nominal events. 
The paper will describe in more detail the operations tested, the CMS tools, and the 
roles and responsibilities of the participants. 

2 METHODS 

The 2011 simulation built on the previous studies using some of the same 
elements (e.g., the airspace), extending the investigation of other elements (e.g., 
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Adding consideration of off-nominal events. Of interest were the 
trategies and how the TRACON team implemented resolutions to 
events. 


2.1 The Simulation: Route Structure, Scenarios and Winds 


The airspace simulated was the terminal area around the Los Angeles 
International Airport (LAX). Figure 1 shows a map of the airspace displaying the 

routes, waypoints, and sector 
boundaries for west-flow 
operations (based on current 
sectors in the Southern 
California TRACON). The 
airspace was comprised of three 
feeder sectors: Zuma, Feeder 
and Feeder South, and two final 
sectors: Stadium and Downe. 
Aircraft in the simulation flew 
OPDs on merging RNAV 
routes to runways LAX24R and 
LAX25L. Path options in the 
form of named RNAV 
arrival/approach transitions were made available inside the TRACON to absorb large 
delays. RNAV go-around routes were designed to enable controllers to use the CMS 
tools to reinsert go-arounds into the arrival flows to the runways. 

Three one-hour scenarios were developed for the simulation with a planned mean 
throughput of 31 arrivals to each runway. The scenarios were built under the 
assumption that aircraft had been delivered to the TRACON meter fixes by en route 
control with nominal schedule errors between 60s early and 30s late. However, due 
to wind forecast errors, these errors between the estimated time of arrival (ETA) and 
scheduled time of arrival (STA) at TRACON entry differed from that range. In 
addition to the standard wake spacing distances an additional buffer of 0.5nmi 
(Nautical Miles) was added into the scheduler. 

Winds were a headwind aligned with the landing runways from 265° a third of 
the time, and the rest of the time were from 45° north or south of the runway. Below 
1,500ft the forecast wind profile matched the actual wind profile, but above this 
altitude there were two wind-forecast-error conditions where the actual wind 
differed from the forecast wind by either 7 or 13 knots. This had an effect on the 
accuracy of the higher-level tools (see below) because their calculations took the 
forecast winds into account. 

Four types of off-nominal event were planned: on-board medical emergencies, 
radio outages (NORDO or “no radio”), pilot-initiated go-arounds (e.g., due to gear 
malfunctions), and tower-initiated go-arounds (e.g., due to another aircraft on the 
runway). Two of these events, of different types, were scheduled in each run. 

The study was run in the Airspace Operations Laboratory (AOL) at the NASA 
Ames Research Center using Multi Aircraft Control System (MACS) software 
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(Prevot, et al., 2010). Simulated aircraft were assumed to be Flight Management 
System and Automatic Dependent Surveillance-Broadcast-out equipped. 

2.2 CMS Tools 

Controller participants worked with a MACS emulation of the Standard Terminal 
Area Replacement System (STARS) onto which the CMS tools were added. As the 
Supervisor’s role was to manage the traffic schedule, his four tools were located on 
two timelines (Figure 2). The two timelines showed schedules to the two arrival 
runways with aircraft ETAs on the left side of the time tape and their STAs on the 
right. The ETA computations and the tools were based on aircraft trajectories being 

predicted through the forecast winds. Green 
bars in the time tape indicated excess spacing, 
or “gaps,” in the schedule and red bars 
indicated insufficient spacing (overlap). The 
Supervisor’s four tools were located in a row 
of buttons near the top of each timeline. He 
was able to “assign” a particular ST A to an 
aircraft by dragging its current STA to a new 
desired time; “swap” the STAs of two aircraft, 
i.e., the STA of aircraft b became the STA of 
aircraft a and vice versa; “reset” all aircraft 
after a specified time which re-scheduled 
aircraft according to the current ETA 
information; and, “move” multiple aircraft 
STAs forward or backward on the schedule by 
a constant amount. These tools are based on 
aircraft trajectory predictions computed using 
the forecast winds. The timelines were located 
on a display ranged out to show the entire 
TRACON as well as about 40nmi into the 
Center airspace (approx lOOnmi radius). 

The controllers also had a suite of four 
tools. These differ from the Supervisor’s tools 
as they are focused on individual aircraft 
rather than the schedule. The CMS controller tools are designed to provide a 
temporal and spatial awareness of each aircraft’s progress relative to its STA, and 
speeds a controller could issue to correct schedule errors (see Kupfer, et al., 2011 for 
a detailed description). 


2.3 Participants and Their Tasks 

Eight retired air traffic controllers participated in this simulation. Three staffed 
the feeder positions and two the final positions. The sixth participant, who served as 
the Supervisor, was recruited specifically for his professional experience as a 
terminal-area traffic manager. Two controllers staffed supporting confederate 




“ghost” and tower positions, and general aviation students and pilots ran eight 
simulation pilot stations to control the simulated aircraft. 

Controllers were asked to manipulate the traffic using only speed and the pre- 
defined path options, if possible, to bring the aircraft to land at the runways on 
schedule, although vectoring was still a valid option. The Supervisor was asked to 
manage the schedule and to try to maintain high runway utilization by working the 
schedule only as far out as the first aircraft that was outside the freeze-horizon (about 
80nmi out from the runways). How the controllers and the supervisor used the tools 
to achieve this and how they coordinated to formulate off-nominal recovery plans 
was a point of research interested and left open for them to determine. 

Participants took part in five days of training during shakedowns prior to the 
study to ensure their familiarity with the concept, tools, and procedures. The study 
ran for two weeks (non-consecutive) during which 46 one-hour runs took place - 42 
runs were unique and four were repeats. Each run had two scripted off-nominal 
events that occurred in three base scenarios under varying wind conditions. The run 
matrix was randomized to reduce controller learning effects. All but one scripted 
event initiated as planned, yielding 91 off-nominal event examples. 

Data were recorded for each run through the MACS’ data collection logs, 
including aircraft and tool states, as well as controller and pilot actions. Screen and 
voice recordings were also collected. Following each run, the participants completed 
an online questionnaire that included questions about their strategies and problem 
solving. In addition, detailed observer notes were compiled from each experimental 
run and debriefing. The following section describes results from analyses conducted 
thus far. 


3 RESULTS 

The Supervisor’s role was to manage the schedule manipulations that were 
required during off nominal events. As this was an exploratory position, the aim of 
the study was to observe the strategies the Supervisor developed and how they were 
executed, and to record his interactions with the rest of the team, who were 
controlling the traffic in the TRACON. 

3.1 Supervisor Strategies 

The Supervisor developed four basic strategies that he often used to begin 
handling the four different types of off nominal event. When a NORDO aircraft was 
identified, the Supervisor typically began by setting its STA to match its ETA 
(using his “assign” tool). He then assessed the schedule and how aircraft ahead or 
behind the NORDO aircraft could be affected. He usually consulted with the feeder 
controller(s) to determine whether to swap STAs, and in some cases devised a 
contingency plan for having the aircraft ahead of the NORDO go around if safe 
spacing was lost. For medical emergencies, the Supervisor coordinated with the 
controllers to expedite the emergency aircraft if possible; the plan could include 
schedule swaps and delaying other aircraft. When go-arounds were declared, the 
controllers needed to formulate a plan for climbing the aircraft to a safe altitude and 
assigning the desired RNAV route to re-enter the aircraft into the flow. The 



Supervisor informed this route choice because he considered the schedule and 
adjustments of neighboring aircraft in the planned sequence. In some cases, the 
Supervisor organized the aircraft to go around to the other arrival runway if its 
schedule had more space. For tower- initiated go-arounds, less time was spent 
developing a plan because the event was announced much later on an aircraft’s final 
approach than for pilot-initiated go-arounds. 

These differences in strategies were reflected in the work that the Supervisor 
estimated he did for each type of off-nominal event (Table 1). For example, he 
worked three times as long, and twice as hard, on NORDO problems as on medical 
events because for medical events he was able to set up a plan and let it play out, 
whereas in a NORDO event the actions of the aircraft were unknown, even though 
they were expected to follow the charted procedure, which resulted in NORDOs 
entailing higher levels of monitoring. 


Table 1. Supervisor’s workload during four types of off-nominal events 



Medical 

Emergency 

Pilot initiated 
go-around 

Tower initiated 
go-around 

NORDO 

Mean estimated task time 

2 min 46 s 

5 min 15 s 

5 min 15 s 

9 min 

Mean estimated task mental 
load 

1.76 

2.87 

2.9 

3.62 

Mean estimated task time 
pressure 

1.66 

2.41 

2.85 
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3.2 Variations to a Plan 

Unsurprisingly, not every strategy played out the way the Supervisor intended. 
Some plans fell victim to wind changes or to other actions that impacted them. For 
example, in Trial 25 the wind shift to a direct headwind pushed the ETAs of all 
aircraft back, making them late with respect to the schedule; while in Trial 45 a 
favorable wind change helped an aircraft advance into its slot marker at an earlier 
waypoint. Some strategies simply followed a different course than the one the 
Supervisor predicted. For example, in Trial 36 a NORDO aircraft kept its speed up, 
forcing the Supervisor to abandon his plan of schedule ’tweaks’ and take an aircraft 
around to the other runway. 

The Supervisor reported he made adjustments to his original plan about 60% of 
the time but these were most often ’a few adjustments’ and were Targe revisions’ 
for less than 10% of the events. Usually he did not adjust a medical event plan 
(median =1 out of 7: no adjustments) but he had to make ‘a few adjustments’ to 
plans for NORDO events (median =3). In a companion question, the Supervisor was 
asked what he would have done differently in hindsight. He reported he would not 
have done anything differently in 73% of the events, but he would have made 
changes on 15 occasions (17%). Half of these reports were after NORDO events 
and, although the Supervisor would usually have done more (Figure 3), on one or 
two occasions he thought a better course of action would have been to do less: “I 
would have let the NORDO run and monitored the situation,” (Trial 36). On four 
occasions he felt his initial plan would have worked and he should not have made 
the additional adjustments that he did. For example, in Trial 17, after initially saying 
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the aircraft with the medical 
emergency should stay at its 
assigned STA, the Supervisor 
decided to swap it with the 
aircraft in front. To achieve this, 
Stadium had to issue multiple 
vectors to both aircraft - a large 
increase in workload. 

The larger number of 
adjustments and plan changes 
(with hindsight) that the 
Supervisor made during the 
NORDO events highlight that 
the NORDO s, in general, were 
the least flexible of the four 
event types. Not only was a 
NORDO inherently less 
predictable, because the pilot 
could not be contacted, it was also less manipulable because the controllers could 
not redirect the problem aircraft and the Supervisor had to adjust the schedule 
around it. 


3.3 Collaboration 

As the team-member with an overall view of the TRACON traffic and the 
schedule, the Supervisor was the natural role to initiate recovery plans for off- 
nominal events. Plans were sometimes formed by the Supervisor alone but often, if 
there was time, he would discuss the situation with the controllers to generate a plan 
that was informed by their views. For example, in Trial 29 the Supervisor 
considered a swap but as part of a discussion with Downe he demonstrated this 
action and Downe said that he would not be able to meet the new STAs. Due to 
this, they began to work on other options. This strategy of consulting the team was 
usually worthwhile because the controllers had to be able to execute the plan that 
the Supervisor created. However, controllers often did far more than merely 
“execute” the plan. In approximately a third of the events the plan would not have 
had the successful outcome that it did if a controller had not paid additional 
attention to the key aircraft and creatively manipulated them into the right place. 
Sometimes these actions were to use the path extension routes or extra speed control 
but at other times the controller solution reflected years of experience; in Trial 28 
Downe vectored one aircraft off the final in a U-shaped detour that allowed another 
aircraft to overtake without having to swap them. 

Sometimes a plan was determined or requested by a controller. Although the 
Supervisor estimated that he generated the plan 60% of the time (Figure 4) and he 
collaborated 30% of the time, these were occasions where the controller suggested a 
distinctly different option (not the creative execution discussed above). Joint plans 
were created most often for tower-initiated go-arounds probably because the initial 
actions for this event - to break an aircraft out of its final descent and assign it the 
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go-around route - had to happen 
quickly. Final controllers often 
assigned a route and then asked 
the Supervisor if it would work 
which began an exchange of 
ideas. 

In the handful of cases where 
there needed to be a new plan 
controllers were involved half 
the time in the Supervisor’s 
revisions. This included asking 
the Supervisor for additional 
help because the STA could not 
be met, reporting that a plan 
would not work for their 
position, and working through 
options to find a viable solution. 


3.4 Interactions of actions 

Given that the Supervisor’s role was a new component of the research and that 
he didn’t have clearly defined procedures to follow, there was a certain degree of 
experimentation as his role organically evolved and he became more comfortable. 
While this was a necessary part of the learning process, it also came with some 
interesting side effects. 

The supervisor tools were all related to adjusting the runway schedule; his 
actions directly changed the ground system’s STA of an aircraft. At the same time 
however, the controllers’ tools were driven by aircraft ST As. Consequently, DSTs 
on the controller displays fluctuated as a direct result of the supervisor’s actions. 
This was usually a good thing: e.g., an aircraft needed to be delayed to make room 
for the re-insertion of a go-around aircraft. In such a case the controller tools helped 
to understand the current state of the aircraft relative to meeting its goal of the new 
STA. On the other hand, there were situations when formulating a plan was difficult 
for the Supervisor and required him to “try out” a few potential ideas before 
finalizing the plan. To do this, the Supervisor simply adjusted the STAs of the 
affected aircraft in a particular way and examined the result (sometimes with and 
sometimes without controller feedback). At that point, the supervisor could have 
kept the plan, reverted to the original state, or tried adjusting the STAs of the 
affected aircraft in a different manner. Even if a plan formulation like this took less 
than a minute, during that time the controller DSTs could change dramatically in 
response to the supervisor’s actions. In multiple instances, a controller saw the 
change to their aircraft’s STA and immediately began working toward the new goal, 
only to discover that it was a “test” rather than the finalized goal. 

A more extreme example shows how the interaction of the Supervisor’s planning 
process and the controllers’ efforts to manage their aircraft can cause undesired 
outcomes. In Trial 35, the NORDO aircraft was a little ahead of schedule, and the 
Supervisor planned to swap the NORDO with the aircraft scheduled just ahead 


(flight HAL5327). After the swap, HAL5327 was ahead of its STA and the Zuma 
controller working the flight issued it a path extension as well as a speed reduction. 
Three minutes went by while the Supervisor examined the changes he made to the 
schedule, after which he determined this plan would not work, and swapped the two 
aircraft back. This now put HAL5327 behind its STA, so much so that Zuma 
informed the Supervisor that even with speed increases the STA could not be met. 
The Supervisor adjusted the schedule a third time, entailing moving the HAL5327 
STA back again, as well as delaying several other aircraft ST As. Zuma slowed 
HAL5327 again, and was somewhat frustrated at having issued multiple 
contradictory speed clearances. He then spent the rest of Trial 35 absorbing sizable 
delays for most of the aircraft flying through his sector. 

In this example, the Supervisor’s delay in solidifying a plan without 
coordination, combined with the actions already taken by the controllers, 
complicated the problem unnecessarily, and resulted in an inefficient solution. 
Interestingly, Trial 35 was repeated as Trial 46, and in Trial 46 the supervisor did 
not attempt to swap the NORDO aircraft with HAL5327, yielding a much smoother 
outcome. Figure 5 shows the lateral tracks of aircraft that flew through the Zuma 
and Stadium sectors during Trials 35 and 46. 



Figure 5. Lateral tracks of aircraft flown through Zuma and Stadium sectors in Trial 35 (left) and Trial 
46 (right). Compared to Trial 46, the tracks in Trial 35 show increased vectoring in both the Zuma and 
Stadium sectors, stemming from the Supervisor’s inefficient plan. 

4 DISCUSSION 

While prior studies of nominal operations (Kupfer, et al., 2011) illustrated little 
need for schedule manipulations, this study of off nominal events clearly shows the 
importance of the role that the Supervisor played. Although his responsibilities were 
pre-defined as monitoring and managing the schedules, the role of ‘strategist’ and 
the procedures for managing events were specifically not defined. During the study, 
the need for a single coordinator role emerged. As the position with an overview of 
the TRACON the Supervisor was able to identify options - such as swapping 
aircraft to a different runway, especially in the go-around cases - that the sector 
controllers were unable to see. In this sense, he became the team strategist as well as 
the schedule manager. During the debrief, when asked who should be in charge of 
creating the plan, the controllers echoed the idea the supervisor was in the best 


position to do so, stating “I knew [the supervisor was] the only one with the big 
picture. I learned that [lesson] early; that I’ve got the small picture.” 

Over the course of the two-week study, the TRACON team developed initial 
strategies for each of the four event types that gave them a going-in position that they 
modified to meet the specific conditions of each trial. These strategies could become 
the basis for developing specific procedural guidance for managing these particular 
types of off-nominal events. Notwithstanding this, the trials in this study provide 
case-studies for comparing the different approaches the team tried and illustrate how 
some approaches were more effective than others at bringing the schedule back to a 
nominal state. Further research is needed to ensure off-nominal recovery procedures 
are specified at the correct level of abstraction and conditions for applying particular 
procedures are clearly defined (Callantine, 2011). It is interesting that although the 
same basic event was initiated multiple times, they rarely played out in the same 
way. 

The functioning of the controllers and the Supervisor as a team unit were 
highlighted by the off-nominal events. During nominal operations there was little 
need for the team to coordinate because the CMS tools provided the required 
information. However, off-nominal events illustrated that supervisor-controller 
communication was key to many of the successful solutions the team identified. This 
is most definitely not a one-way channel; controller input was valuable to the 
Supervisor in many cases. Not only do the controllers have a detailed view and 
understanding of the situation that a supervisor does not have but in this study, he 
needed their feedback to assess whether he had solved a problem or just shifted it. A 
“ripple” effect was observed for some Supervisor actions where workload increased 
in sectors that were not involved with the off-nominal problem (usually the East 
Feeder). 

As seemingly small actions on the schedule can have large and often 
unforeseeable consequences, off-line planning tools are key. Problems were observed 
that were complicated by the Supervisor trying a solution to look at its effects and 
reversing his actions but not before the controllers had begun to issue clearances 
based on the change. The team tried to develop a workaround where the Supervisor 
announced when he was planning and when he had set the schedule, but this 
frustrated the controllers as their tools reset and they had to wait to make 
adjustments. The Supervisor needed a ‘schedule trial-planning’ function so he could 
assess the implications of his plans before he set them as changes into the ground 
automation. A pre-existing MACS schedule-adjustment functionality could be 
streamlined and extended to enable schedule trial-planning to meet this requirement 
in future studies. Further analyses to determine whether any of the Supervisor’s tools 
could or should be automated also need to be conducted. 


5 CONCLUSION 

This study investigated the robustness of a schedule-based arrival-management 
concept using controller tools and introduced a new supervisory position to manage 
the schedule under off-nominal events. The controller team successfully managed 
most of the trial cases during a two-week study, suggesting that this concept has 
potential to demonstrate consistently robust performance even during off-nominal 
events. The results indicate recovery from off-nominal events is most efficient when 



rescheduling is accomplished reasonably quickly and the TRACON team is able to 
use the tools to make the required adjustments. The larger number of adjustments 
that the Supervisor made during the NORDO events and the greater number of 
actions he would have changed with hindsight illustrate that the NORDO events, in 
general, were the hardest, and most workload-intensive to manage. The case studies 
have provided rich information about possible strategies for development of off- 
nominal recovery procedures, and have shown where improvements to the supervisor 
tools are required and where the study can be improved for future investigations. 
While the research is a first step toward establishing the necessary safety case for 
CMS operations, the results are promising, and should help pave the way for future 
development of controller tools, procedures, and simulation-capabilities. 
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